(1) Field of the Invention
The present invention relates to an image processing apparatus and an image processing method for generating a reconstructed image where mosquito noises that occurred as a result of block coding are effectively eliminated without blurring the edges and textures in an image by performing filter processing on the decoded image.
(2) Description of the Related Art
It is becoming popular to take an image (still image) such as a scenic shot or a people photograph using a digital still camera that has become widespread in recent years. However, the data amount of a digitized image is huge. Storing the image with huge data amount on a storage medium such as an IC memory as it is or transmitting it using a transmission medium such as the Internet or LAN causes the result that only a few images can be barely stored on a storage medium or the transmission time in a transmission medium becomes longer. Therefore, image compression technique is necessary in handling images.
When there is a need to compress image data at a high ratio, an irreversible compression method where the decoded image does not completely match the original image is generally used. Most irreversible compression methods employ a coding method of dividing an image data into blocks of M×N pixels, performing an orthogonal transform on the respective blocks, and then quantizing the orthogonal transform coefficient. One of these representative methods is JPEG that is widespread as a color still image compression method.
FIG. 1 is a block diagram showing the functional structure of the JPEG coding apparatus and the JPEG decoding apparatus. The coding apparatus 60 compresses the original image into the JPEG-compressed data, and it comprises a pre-processing unit 61, a DCT (discrete cosine transform) transformation unit 62, a quantization unit 63, a quantization table 64, and an entropy coding unit 65 as shown in FIG. 1. The decoding apparatus 70 decompresses the JPEG-compressed data to a decoded image, and it comprises an entropy decoding unit 71, a dequantization unit 72, a dequantization table 73, an IDCT (inverse discrete cosine transform) transformation unit 74, and a post-processing unit 75 as shown in FIG. 1.
When the JPEG coding processing is performed in the coding apparatus 60, the pre-processing unit 61 transforms each pixel data made of multi-valued data of R (Red), G (Green) and B (Blue) of the original image into data of Y (luminance component) and Cr and Cb (color difference components) first.
Next, the DCT transformation unit 62 performs the DCT on each 8×8 pixel block of YCbCr data, and calculates the DCT coefficient.
Next, the quantization unit 63 performs quantization of the DCT coefficient. Each component of the DCT coefficients is quantized using a different step width according to the quantization table 64 at this time.
Lastly, the entropy coding unit 65 performs coding of the quantized DCT coefficient and generates the JPEG-compressed data. The JPEG standard method employs the Huffman coding as an entropy coding.
The above-mentioned processing is the outline of the coding processing from image data to the JPEG-compressed data. The JPEG-compressed data generated in this way is delivered to the decoding apparatus 70 via a storage medium (such as an SD card) or a transmission medium.
Next, processes of the decoding processing from the JPEG-compressed data to a decoded image will be explained.
When the JPEG decoding processing is performed in the decoding apparatus 70, the entropy decoding unit 71 performs entropy decoding on the JPEG-compressed data first.
Next, the dequantization unit 72 performs dequantization. At that time, the dequantization unit 72 reads out the information on the quantization table 64 used at the time of coding from the JPEG-compressed data and uses it as the dequantization table 73.
Next, the IDCT transformation unit 74 performs the IDCT and transforms the DCT coefficient into a decoded image of YCbCr data.
Lastly, the post-processing unit 75 obtains a decoded image by performing a transformation processing from the YCbCr data to RGB data.
The above-mentioned processing is the outline of the coding processing and the decoding processing concerning the JPEG.
As mentioned above, quantization of the DCT coefficient is included in the process of the JPEG coding processing. Therefore, data degradation is caused by the quantization errors. As a result of this, this degradation appears as noises on a decoded image when reproducing the decoded image on a paper using a printer or the like as it is. Even small noises on a still image can be obtrusive because still images can be viewed for a longer time, and these noises are also obtrusive in the case of moving image compression for performing block coding.
There is a kind of noise that exerts bad visual influence on a decoded image. This kind of noise is called mosquito noise. Mosquito noise means fuzzy dots that occur around the edges of decoded images. This results from the incapability of reproducing strong edges which existed on the original image to be reproduced accurately because a lot of high-frequency components are lost by quantization of the DCT coefficient.
In order to eliminate this kind of noise, a filtering processing is performed on the decoded image, and the image processing apparatus 100 for generating a reconstructed image where mosquito noises are eliminated has been conventionally considered as shown in FIG. 2. A lot of filter processing is proposed as a filter processing method, and the SUSAN filter shown in formula (1) is one of the new filters capable of keeping edges sharp and eliminating noises (for example, refer to the non-patent literature 1). Note that this technique is also called “the first prior art” from here.
                              g          ⁡                      (                          x              ,              y                        )                          =                                            ∑                              i                ,                j                                      ⁢                          {                                                f                  ⁡                                      (                                                                  x                        +                        i                                            ,                                              y                        +                        j                                                              )                                                  ·                                                      α                                          x                      ,                      y                                                        ⁡                                      (                                          i                      ,                      j                                        )                                                              }                                                          ∑                              i                ,                j                                      ⁢                          {                                                α                                      x                    ,                    y                                                  ⁡                                  (                                      i                    ,                    j                                    )                                            }                                                          (        1        )            
whereαx,y(i,j)=α1x,y(i,j)·α2x,y(i,j)  (2)α1x,y(i,j)=exp [−(i2+j2)/(2σ2)]  (3)α2x,y(i,j)=exp [−{f(x+i,y+j)−f(x,y)}2/t2]  (4)
Here, in the formulas (1)˜(4), (x, y) shows the position of the current pixel, (i, j) shows the relative position of each peripheral pixel to the current pixel, f (x, y) shows the pixel value of the current pixel in the decoded image, f (x+i, y+j) shows the pixel value of each peripheral pixel in a decoded image, and g(x, y) shows the pixel value of the current pixel after filtering processing. Also, αx, y (i, j) shows the filter coefficient of each peripheral pixel (i, j) to the current pixel (x, y), and the filter coefficient is determined by the product of the first filter coefficient α1x, y (i, j) and the second filter coefficient α2x, y (i, j). Here, the first filter coefficient α1x, y (i, j) is designed so that the filter coefficient of a peripheral pixel increases in response to getting nearer to the current pixel, and the second filter coefficient α2x, y (i, j) is designed so that the filter coefficient of a peripheral pixel increases in response to its pixel value getting nearer to the pixel value of the current pixel.
Note that σ is a parameter for adjusting the influence of the distance from the current pixel to each peripheral pixel on the first filter coefficient α1x, y (i, j), and t is a parameter for adjusting the influence of the differential value between the pixel value of the current pixel and the pixel value of each peripheral pixel on the second filter coefficient α2x, y (i, j).
The biggest feature of this filter is the calculation method of the second filter coefficient α2x, y (i, j).
Here, FIG. 3 shows the relation between the differential value between the pixel value of the current pixel and the pixel value of each peripheral pixel, and the value of the second filter coefficient α2x, y (i, j) that is calculated from this differential value.
As is evident from this FIG. 3, a big value is set as the second filter coefficient α2x, y (i, j) when the pixel value of a peripheral value is close to the pixel value of the current pixel, on the contrary, a small value is set as the second filter coefficient α2x, y (i, j) when the pixel value of a peripheral value is widely different from the pixel value of the current pixel.
Here, a strong smoothing is performed when a big value is set as parameter t because a big filter coefficient is set in the case of a peripheral pixel whose pixel value is widely different from the pixel value of the current pixel. On the contrary, a weak smoothing is performed when a small value is set as parameter t because a small filter coefficient is set in the case of peripheral pixels except the peripheral pixels whose pixel values are very close to the pixel value of the current pixel. Therefore, there is a need to preset the optimum constant value as parameter t according to the purpose of the processing when applying the SUSAN filter.
The case for eliminating mosquito noises using this SUSAN filter will be considered.
As mentioned earlier, mosquito noises occur as fuzzy dots around strong edges.
FIG. 4 is a diagram schematically showing the relation between edges and mosquito noises. Note that the value of parameter t of the SUSAN filter should be set well smaller than the tone changes on edges and well bigger than the tone changes of mosquito noises. Here, because a very small filter coefficient is set on pixels including edges that have a pixel value widely different from the pixel value of the current pixel by the effect of the second filter coefficient α2x, y (i, j) without exerting much influence on the filter processing in effect when applying the SUSAN filter on pixels with mosquito noises, filter processing is performed setting a big filter coefficient on pixels with mosquito noises around the current pixel that have a pixel value close to the pixel value of the current pixel. Therefore, it is possible to keep edges sharp and eliminate mosquito noises effectively.
Considering this, filtering processing where filter coefficients are determined based on the differential value between the pixel value of the current pixel and the pixel value of the peripheral pixel is very effective so as to eliminate mosquito noises keeping the edge sharp.
Also, the following image processing apparatus has been conventionally considered (refer to patent literature 1): an image processing apparatus for eliminating noises from image data divided into a plurality of blocks block by block, and it comprises a judging unit operable to calculate frequency distribution of differential value between the neighboring pixels block by block based on image data for respective blocks and judge whether any edge is included as to each block based on the frequency distribution and a selection unit operable to select blocks where noises should be eliminated based on the result judged by the judging unit. Note that this technique is also written as “the second prior art” from here.
This second prior art makes it possible to eliminate mosquito noises and keep textures sharp by a simple processing because filter (E filter) processing is performed on blocks including edges and no filter processing is performed on blocks without edges as shown in FIG. 5.
[non-patent literature 1]
S. M. Smith and J. M. Brady, “SUSAN—A New Approach to Low Level Image Processing, “International Journal of Computer Vision, vol. 23, no. 1, pp. 45-78, 1997.
[patent literature 1]
U.S. Pat. No. 3,011,828 (FIG. 1 in the first page)
However, according to the first prior art, when using the SUSAN filter so as to eliminate mosquito noises, a distinctive effect of keeping edges sharp even when eliminating mosquito noises that occurred in the sign area of the signboard is obtainable as shown in FIG. 6, but there occurs a problem of causing great degree of blur in the texture area such as the area of leaves. This comes from eliminating the tone change of the texture that existed in the original image when performing filter processing on the texture area using this parameter t because there is a need to set the value of parameter t to a value that is much bigger than the degree of the tone change of mosquito noises so as to eliminate mosquito noises effectively.
On the contrary, according to the second prior art, it is possible to keep textures sharp without affecting the effect in eliminating mosquito noises, but there occurs a new problem that mosquito noises are not completely eliminated or that image quality excessively changes depending on the existence of an edge because no filter processing is performed on dull edges.